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Abstract. In this paper we demonstrate how the geometrically 
motivated algorithm to determine whether a two generator real 
Mobius group acting on the Poincare plane is or is not discrete can 
be interpreted as a non-Euclidean Euclidean algorithm. That is, 
the algorithm can be viewed as an application of the Euclidean divi- 
sion algorithm to real numbers that represent hyperbolic distances. 
In the case that the group is discrete and free, the algorithmic pro- 
cedure also gives a non-Euclidean Euclidean algorithm to find the 
three shortest curves on the corresponding quotient surface. 



1. Introduction 

The problem of determining whether a two generator real Mobius 
group acting on the Poincare plane (hyperbolic two-space) is or is not 
discrete is an old one. If such a group is non-elementary and dis- 
crete, it is, of course, a Fuchsian group. There are many approaches 
to this problem, some of them incomplete. One of the most complete 
answers is given by the use of an algorithm. The algorithm can be 
given in a number of different forms [2], including a geometric form 
and an algebraic form. Revisiting the algorithm has been productive, 
as the algorithm has been shown to have a number of useful impli- 
cations, including results about primitives and palindromes in rank 
two free groups, discreteness criteria for complex Mobius groups acting 
on hyperbolic three space, and the computational complexity of the 
discreteness problem [H [21 [U El [7J [10]. In this paper we revisit the 
Gilman-Maskit algorithm [8] in the case of a pair of hyperbolic gen- 
erators A and B with disjoint axes and illustrate that it is a type of 
non-Euclidean Euclidean algorithm. We refer to the Gilman-Maskit 
algorithm as the GM algorithm. 

An algorithm is termed a non-Euclidean Euclidean algorithm if it 
involves performing Euclidean algorithm type calculations to quantities 
that are non-Euclidean lengths, in particular to the translation lengths, 
Ta and Tq, of the isometries when they (or a conjugate pair) act as 

hyperbolic isometries on the Poincare plane. 
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Figure 1 . In calculus one often asks students to analyze 
the picture of a person whose shadow is being cast by a 
lamppost. The analysis can be requested under varying 
rates (e.g. the rate that the person walks away from the 
lamppost or the rate that shadow grows) with different 
quantities shown in the figure known and unknown. Now 
analyze this situation when the distances are all hyper- 
bolic distances and the rates are also rates of change of 
the hyperbolic quantities. 

The algorithms in [H [3] can also be viewed as algorithms to find 
the three shortest geodesic when the group is discrete. The paper [3] 
addresses the case of intersecting hyperbolic axes. We point out how 
the concept of the non-Euclidean Euclidean algorithm applies to all 
types of pairs of isometries acting on hyperbolic two-space as such a 
shortest length algorithm, even in the case where no algorithm is needed 
to determine discreteness. This is done in section El 

A quick heuristic way to understand the term Non- Euclidean Eu- 
clidean algorithm (an NEE algorithm) is by referring to the description^] 
given by Ryan Hoban [9] (see Figured]). 

1.1. The organization of the paper. The main theorems, Theorem 
12.31 and its companion Theorem 12.41 are stated in section [2] using a 
minimal amount of notation and background material. In section [3] we 
review terminology. This includes subsections on notation, factoring, 
and the algorithmic paths. It also includes 13.31 where we give a bare- 
bones description of the algorithm together with figures that illustrate 
the geometry of some of the cases we need to consider. The proofs of 

1 After this interpretation of the algorithm as a NEE algorithm was in place, but 
before the manuscript was written, the author saw this description in a lecture by 
Hoban. 
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the main theorems along with the relevant lemmas and their proofs are 
in section HI At the end of this section, section HJ Theorem 12.31 and its 
companion Theorem 12.41 are combined into a single theorem, Theorem 
14.41 In section [5] the analogy is explained and in section [6] elliptic el- 
ements are addressed and the result on shortest curves, Corollary 16. 1[ 
is stated and proved. 

We begin by stating Theorem 12.31 the main result. 

2. The main result 

We let G be the group generated by A and B both elements of 
PSL(2, M.) and assume throughout this paper that G is non-elementary. 
The results of [3 [21 [3], H] can be summarized as 

Theorem 2.1. [SI H] Geometric Algorithm 

Given A and B there is a an integer t and a set of integers [n\, ...,nt] 
such that replacing the ordered pair (A, B) by the sequence of genera- 
tors: 

(A,B) -> (B~\A~ l B ni ) -> (B~ ni A,A- 1 B ni (A- 1 B ni ) n2 ) -»...(C,£>) 

after t steps gives an ordered pair of stopping generators (C, D) and 
outputs G is 

(i) discrete 

(ii) not discrete, or 
(Hi) not free. 

We note that in the case that G is not free, the algorithm determines 
discreteness or non-discreteness and finds stopping generators, but the 
sequence of integers [nx, ...,n t ] must be modified (see section [6]). The se- 
quence [nx, ■•■,nt] was known as the Finonacci/non-Fibonacci sequence 
of the algorithm [2], [10] and used in the calculation of computational 
complexity. 

Definition 2.2 ( [H [2j [10l H] ) . The sequence [ni,...,n t ] is termed the 
F-sequence or the Fibonacci sequence of the algorithm. 

We first state the main result in the case that the initial and stop- 
ping generators are hyperbolic isometries with disjoint this 
is the most complicated case of the algorithm and then give exten- 
sions allowing parabolics; in section [6] we indicate how this extends to 
all other type of pairs of initial and stopping generators. We use T x 
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and Kx to denote the translation length and multiplier of an element 
X G PSL(2, M) or equivalently the translation length and multiplier 
of a lift of X to SL(2,M.) as defined in section |3~T| where Tr(X), the 
trace of X or a lift is also defined. Note that K x -i = K^ 1 . 

Theorem 2.3. (hyperbolic-hyperbolic initial and stopping generators) 
Assume that A and B are a pair of hyperbolics with disjoint axes and 
the algorithm stops with such a pair. 

If one applies the Euclidean algorithm to the non-Euclidean transla- 
tion lengths of the generators at each step, the output is the F-sequence 
[n u ...,n k ). 

In particular if the multiplier of A is and the multiplier of B is 
Kb, then 

_ A\\ogK A \)/2 
1 [ (|log^ B |)/2 J 

where [ ] denotes the greatest integer function and \ \ absolute value, 
or equivalently if Tx is the translation length of X: 



Tb/2 



and 



and 



n 2 = where D = B~ ni A 

Td/2 



JcJ2- 

TIa 



3 l T Dj /2 } 

where (Cj, Dj) is the pair of generators at step j, 1 < j <t in Theorem 

E3 

Now we extend this to statements of the theorem if the algorithm 
encounters parabolics. Note that a parabolic isometry does not have 
a multiplier. For any X and Y, let [X, Y] be their multiplicative com- 
mutator. 

Theorem 2.4. (parabolic elements) 

Assume that at step t — 1, D t _i is parabolic with C t -i hyperbolic. Let 
A = Ct-i and B = D t -i- Then 

r 2 ■ \Tr(A)\ - 2, 



V\Tr([A,B])\ y 



5 



or equivalently, 

= 4co 3 h ( y-2 

L ~ ,/-/fdnl\ J 



If C t -i and D t _i are both parabolic, n t = 1. 

If one of C t or D t is elliptic, the group is either not free or not 
discrete. 

3. Preliminaries 

3.1. Notation and Terminology. Let X E SL(2, M)/±id denote any 
pull-back of X from PSL(2, M.) = SL(2, M.) / ±id. X acts as a fractional 
linear transformation on the complex plane and acts an isometries on 
the upper-half plane model of hyperbolic two-space endowed with the 
Poincare metric. Since X and X induce the same isometry, we use 
them interchangeably. As conjugates of X and X act as isometries 
on the unit disc model, we move from the upper-half-plane model to 
the unit disc model according to convenience using X for the element 
acting in either model. 

We remind the reader of the classification of isometries algebraically 
by the absolute value of the trace of the pull back. The matrix X 
and the isometry induced by X or X is hyperbolic, elliptic or para- 
bolic according to whether |Tr(X)|, the absolute value of the trace of 
the matrix, is > 2, < 2 or =2. If we begin with A and B in 
PSL(2,18L) and choose A and B to have positive trace, then the trace 
of every element X in the group they generate is determined and we 
set Tr(X) =Tr(X). 

If X is hyperbolic, it fixes two points on the boundary of the unit 
disc and the hyperbolic geodesic connecting the two fixed points known 
as the axis of X and denoted Axx- One of the fixed points is attracting 
and one is repelling, means that X moves points along the axis of X 
toward the attracting fixed point and it moves all points on Axx a 
fixed distance in the non- Euclidean metric, its translation length, Tx- 

Recall that for X hyperbolic, the transformation X is conjugate to 
z (—7- Kz for some real number K > 0, known as the multiplier and 

|Tr(X)| = \/K + \/K . We write Kx for the multiplier of X and 
note that T x = T x -i = | \ogK x \ via cosh = ^Tr(X). 

3.2. Factorization. Now X can be factored in many ways as the prod- 
uct of any two half-turns. In the hyperbolic case this is as the product of 
any two half-turns about perpendiculars to Axx that intersect Axx at 
a distance of ^ apart. If A and B are any two elements of PSL(2, M), 
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Figure 2. Axes of A and B and their common perpen- 
dicular L and some possible configurations for the La and 
Lb lines. In the second case the group will be discrete 
and free. In the third case, A~ l B is elliptic and in the 
last figure the axes of AB~ l , the common perpendicular 
to La and L#, is shown. 

their axes have a common perpendicular L and there are lines La and 
Lb such that A = Hl o Hl a and B = Hl o Hl b . A half-turn Hp 
about a geodesic line P is simply a transformation of order two that 
fixes the line and interchanges its end points on the boundary of the 
unit disc (see Figure [2]). Elliptics and parabolics still have a so called 
axis (sometimes a degenerate line) and can again be factored using the 
common perpendicular. 

3.3. The Barebones Algorithm. Assume we begin with A and B 
hyperbolics with disjoint axes and orient L so that it points from Ax a 
to Axb when Tr(A) > Tr(B) and the attracting fixed points of A and 
B are both to the left of L. 

The algorithm considers the possible configurations for La and Lb- 
For example if these two geodesies intersect then A~ l B is either elliptic 
or parabolic and the algorithm moves up into what is considered to an 
easier case, where the generating pair is either a hyperbolic-elliptic 
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pair or a hyperbolic-parabolic pair. If the lines are disjoint and no one 
separates the other from L, then the three axis L, La and Lb bound a 
region and the algorithm stops indicating that the group is discrete. 

In Figure [2] we show some, but not all of the possible configurations 
for the Lb and La lines. We emphasize those that are relevant to our 
discussion, for example, if La^Lb 7^ and lies interior to the unit disc, 
then A~ 1 B is elliptic so the group even if discrete, is not free. If these 
lines intersect on the boundary of the disc, then A~ X B is parabolic. 

If L B intersects Ax a and L A intersects Axb, we need to know for 
how many q does L B i intersect Ax a where B q = H L H Lm . Analysis 
of this situation is carried out in Section S] (see Lemma 14.11) . This is 
termed the really bad case and is the focus of most of our attention. 
Figure [3] (page 13) illustrates the case when q = 3 and shows portions 
of the axes of B~ X A, B~ 2 A, and B~ 3 A. 

3.4. The algorithmic path. The algorithm considers the different 
possible types of generating pairs and breaks them down into two cases. 
One is the case where both generators are hyperbolic and their axes 
intersect [3]. The other case where the axes of the hyperbolic are 
disjoint is known as the intertwining case [8]. 

In the intertwining case, the algorithm tests the generators for dis- 
creteness and non-discreteness. If neither is found to be true, the al- 
gorithm declares the pair indeterminate and produces a next pair of 
generators. The algorithm returns normalized ordered pairs (termed 
coherently oriented pairs) as the next pair so that we always have 
Tr(Cj) > Tr(Dj) at the j-th step. An implementation of the algo- 
rithm will begin and end with a pair in any part of the possible paths 
given below, but the next pair will follow the path staying stationary or 
moving to the right, allowing at most a finite number, rij, of repetitions 
of a given pair- type before it moves to the right. 

Let H,P and E denote respectively a hyperbolic, parabolic, or elliptic 
generator. The types of pairs in the intertwining algorithmic path are: 

(1) HxH^HxE^PxE^ExE 

(2) HxH^HxP^HxE^PxE^ExE 

(3) HxH^HxP^PxE^ExE 

(4) HxH^HxP^PxP^PxE^ExE 

4. Configurations and Distances: Main Lemma 
We turn our attention to the really bad case. 
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Lemma 4.1. (Main Lemma) // (A, B) satisfy Tr(A) > Tr(B) > 2, 
L A n A%b 7^ 0, and L B n AX a ^ 0, then there exists a positive integer 
n such that 

(T B ) T A . T B 

n < — < in + 1 — . 

2 ~ 2 ~ v ' 2 

Lemma 4.2. If (A, B) satisfy Tr(A) > Tr(B) > 2 andL A f]Ax B = 0, 
fritt PI AX A 7^ ; t/ien there exists a positive integer n such that 

2 - 2 

Lemma 4.3. IfTr(A) > Tr(B) = 2, with L B n Ax a ^ 0, f/ien f/iere 
is a positive integer q such that L B i H Ax^ 7^ 0, £m£ L B q+i fl L^ = 0. 
and g = [ 2 ;l Tr ^)l-2 1 r = Tr(B) = 2, then if L A nL B = */*e 

group is discrete and if L B n L^ 7^ 0, #ien t/ie group is either not free 
or not discrete. The F -sequence ends. 

Proof. (Proof of Lemma 14.11 ) The distance between L and L B along 
the axis of B is T B /2. We consider L B 2,L B 3, L B r-i, L B r successive 
perpendiculars to the axis of B that are each a distance T B /2 from the 
preceding one along the axis of B and note that Hl bt o Hi = B r and 
H Lgr o i^L ss = B r ~ s . The integer of the F— sequence is the r such 
that L B r-i separates L and La, but L B r does not. 

Further if for a given k L B k and L B k+i both intersect the axis of A, 
then the distance between their intersection points along the axis of A 
is greater than T B /2. Distances along the Axis of B are additive, so 
that the distance between L and L B k along the axis of B is kT B /2 and 
if r is the last integer such that L B r intersects the axis of A between L 
and La, then rT B < T A - 

On the other hand, since La intersects Ax B between L B r and L B r+i, 
we have T A < (r + 1)T B . 

Finally, suppose L B r meets Ax^ at L^. Then AB~ r is elliptic so the 
group is not discrete and free. If La meets Ax B at L#r, then again the 
group has an elliptic element and is not discrete and free. □ 

The proof of Lemma I4T21 only requires minor modifications from that 
of Lemma |4~T1 and the integer rij at step j is thus the 

Proof. (Proof of Lemma [4.31 ) If the algorithm begins or encounters a 
parabolic, the final n t has a different definition. At each algorithmic 
step (see [Sj or [2]) the trace of the hyperbolic in the pair (A, B) where 
A is hyperbolic and B parabolic is reduced by a fixed amount, which 
can be computed to be y/\Tr([A, B])\. That is, normalize the matrices 
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by conjugation so that A(z) = z + t and B(z) = where a 2 — b 2 = 1 
to see that the amount by which the trace is reduced can be written as 
\t • b\. Then nt is as in the statement of the Lemma. If both A and B 
are parabolics, then either the product is elliptic and the group is not 
free or the product is hyperbolic or parabolic and the group is discrete. 
Thus n t = 1 and can be omitted. □ 

We note that if Ta = Tg, then after possibly interchanging A and 
B and replacing A and/or B by its inverse, we may assume that Lb 
separates L and La and we are in the case of one of the first two 
lemmas. 

Proof. (Proof of Theorem 12.31 ) Assume that at some point the al- 
gorithm returns the pair (A, B). If La and Lb intersect, then the 
algorithm is stopped and G is either not free and or not discrete. If 
the three geodesies L, La and Lb bound a region, G is discrete. If 
J0rgensen's inequality is violated, G is not discrete. Otherwise, we are 
in one of the cases of one of the three lemmas and the result follows. □ 

Combining the results we have 

Theorem 4.4. Let G = (A, B) and let (Ct,D t ) be the stopping pair 
for the GM algorithm. 

(1) Assume that A and B are a pair of hyperbolics with disjoint 
axes and that the GM discreteness algorithm stops with such a pair. 
If one applies the Euclidean division algorithm to the non-Euclidean 
translation lengths of the generators at each step, the output is the F- 
sequence [ni, ...,n t ]. 

In particular if the multiplier of A is Ka and the multiplier of B is 
Kb, then 

_ (|logK A l)/2 

1 [ (|logi^|)/2 J 

where [ } denotes the greatest integer function and | | absolute value, 
or equivalently if Tx is the translation length of X: 

r?V 2 i 



Hi 



"TbIT 



and 



and 



n 2 = where D = B~ ni A 



l T D /2 3 



3 [ T D J2 1 
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where (Cj, Dj) is the pair of generators at step j, 1 < j <t in Theorem 

(2) If at step j (Cj,Dj) is hyperbolic-parabolic pair, then j = t—land 

2.|7V(C t -0|-2 



ViTr([a-i,A-i])r 

or equivalently, setting A = C t -\ and B = D t _i. 

4cosh(^f) - 2 



n t - 

2 



2cosh(toi) 



(3) If at step j (Cj,Dj) is a parabolic-parabolic pair, j = t and 
n t = l. 

Fort > 2, at step (t — 2), (Ct-2, Dt-2) is a hyperbolic-parabolic pair 

and = [ 2-\Tr(C t - l)h 2 j 

(4) If the initial pair is a hyperbolic-parabolic pair, then the F- 

4cosh( — —)— 2 

sequence of length 2 and is [^1,77.2] where n\ = [ T 2 — ] andn2 = 1. 

2 cosh ( I a ' ' ) 

// the initial pair is a parabolic-parabolic pair, the F-sequence is of 
length 1 with n\ = 1 . 

5. The analogy: the Euclidean and the non-Euclidean 

algorithm 

We find the sequence [ni,...,n t ] of the discreteness algorithm by a 
combination of Euclidean or division algorithm type of computations 
with hyperbolic lengths and hyperbolic length replacements with the 
remainder term (in keeping with the trace reducing aspect of the algo- 
rithm). 

Let a = (| log K A \)/2 and b = (| log K B \)/2. 

Then these are simultaneously hyperbolic lengths and real numbers. 
We can do the first step of the Euclidean type algorithm on a and b to 
obtain (assuming a > b): 

Step 1. a = nib + b\ where < bi < b and n\ is a positive integer. 

Step 2. At step 2, in a standard Euclidean algorithm, we would 
normally work with b\ and b: 

BUT we replace bi by Tjj/2 = | logi^ol/2. (Note: we know from 
the geometry To/2 < 61.) 

That is, set D = B~ ni A and use b\ = T D /2. 



11 



Subsequent Steps. Next, b = n?b\ + 62 where < 62 < b\ and 712 
is an integer. Replace 62 by 62 = Te/2 where E = B(A~ 1 B ni ) n2 and 

continue. Note that T E = T E -i. 

Stop. The geometric proof that the algorithm stops in a finite num- 
ber of steps (b/c the trace is reduced at each step by at least a minimal 
amount) says that here after a finite number of steps, we are at a stop- 
ping point. This is recognized by a trace that is less than 2 (as the 
initial traces are taken to be positive). Thus the algorithm stops if 
either the group contains an elliptic (trace between -2 and 2), the axes 
bound a region, or J0rgensen's inequality has been violated. 



6. Including elliptics and intersecting axes 

In the case that the algorithmic path encounters an elliptic, either 
the elliptic is of infinite order in which case the group is not discrete or 
it is of finite order in which case the algorithm proceeds until it reaches 
a decision about discreteness. The F-sequence needs to be modified 
(see (TTJ) because the algorithm uses both hyperbolic distances and 
angles. Angles are the same for Euclidean and non-Euclidean geome- 
try. The interpretation as a non-Euclidean Euclidean algorithm can be 
continued, but we do not do so here. 

For hyperbolics with intersecting axes, one does not need an algo- 
rithm to determine discreteness unless the commutator is elliptic. How- 
ever, the same rule for replacing generators using the F-sequence will 
stop and find the shortest generators. 

Thus whether we are in the intertwining case or the intersecting axes 
case, if the group is discrete so that the quotient is a surface, the steps 
used in the algorithm can be applied and interpreted as an algorithm 
to find the three shortest geodesies on the quotient (see [U, [5]). This 
algorithm can be interpreted as a non-Euclidean Euclidean algorithm 
with the same definitions of the rij. We have: 

Corollary 6.1. (Shortest Curves) (initial generators hyperbolics 
with intersecting or disjoint axes) Given G = (A, B) C PSX(2,M) 
where A and B are hyperbolic either with disjoint or intersecting axes, 
there is an non-Euclidean Euclidean Algorithm that finds the generators 
corresponding to the three shortest curves on the quotient surface when 
the group is discrete. There is an F -sequence of integers and these 
integers are calculated by the same formulas as the formulas in Theorem 
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Figure 3. A number of Lb lines with La intersecting 
the axis of B. Dotted green lines represent L B , L B 2, 
L B a,L B 4 respectively. Axes of A, B~ l A, B~ 2 A,a,nd B~ 3 A 
are solid red lines. 



